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SYSTEM AND METHOD FOR MEASURING AND MANAGING PERFORMANCE IN 
AN INFORMATION TECHNOLOGY ORGNIZATION 

Background of the Invention 

Technical Field of the Invention 

This invention pertains performance management. More 
particularly, it relates to measuring and managing the 
performance of an information technology (IT) organization. 

Background Art 

Developing a system that measures the performance of an 
IT organization is critical to managing the IT organization 
and continually improving the processes that it employs over 
time. However, no regular, canonical method for developing 
a system that measures the performance of an IT organization 
is evident in the industry. The lack of such a method 
creates great difficulty in understanding the performance of 
the IT organization, and this is a critical aspect to 
successful management and improvement of the IT organization 
over time. This problem has existed since the formation of 
IT organizations. Consequently, there is a need in the art 
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for a repeatable work product-based method for defining and 
implementing a measurement program for an IT organization. 

There is a need in the art for a system and method for 
providing measurement gap analysis describing the 
relationship between the measurement system and the 
underlying organization and processes. Without appropriate 
organizational and process support, it is difficult for any 
measurement system to achieve maximum potential usefulness. 
Measurement systems that are supported by the appropriate 
organizational structure and are fully integrated with the 
organizations process have a much higher probability of 
being effective. 

It is an object of the invention to provide a system 
and method for systematically defining and implementing a 
measurement program. 

It is a further object of the invention to provide a 
system and method for implementing a measurement program 
that is supported by an appropriate organizational structure 
and that is fully integrated with the organization's 
processes • 



It is a further object of the invention to provide a 
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system and method for identifying and defining measurements 
necessary to support the management of an IT organization. 

It is a further object of the invention to provide a 
system and method for linking defined measurements to 
organization goals and objectives. 

It is a further object of the invention to provide a 
system and method of integrating measurement ownership into 
an organizational model. 

It is a further object of the invention to provide a 
system and method for integrating measurement data 
collection into an IT process model. 

It is a further object of the invention to provide a 
system and method for developing a technical environment for 
supporting measurement data collection, analysis and 
reporting. 

It is a further object of the invention to provide a 
system and method for developing the technical interfaces 
necessary to support measurement data collection. 



It is a further object of the invention to provide a 
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system and method for developing measurement reporting 
methods . 

It is a further object of the invention to provide a 
system and method for developing training and end-user 
support materials for a measurement system. 



Sxammary of the Invention 



A system and method for defining and implementing 
measurements to support a customer information technology 
organization. Steps involve identifying customer 
performance goals; building a model in response to customer 
goals including a plurality of primitive metrics; performing 
a gap analysis of the model to determine which of the 
primitive metrics are already collected by the organization 
and the process capabilities to support the data collection; 
identifying new data collection sources for the primitive 
metrics which are not already collected by the organization; 
and implementing the tools and processes for gathering all 
of the primitive metrics and tools for generating 
measurement reports from the gathered primitive metrics. 
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In accordance with a preferred embodiment of the 
invention, measures of performance of a customer information 
technology organization are determined by defining the 
organization goals, identifying the behaviors that the 
organization exhibits when working toward or meeting the 
defined goals, identifying and defining the measurements 
that provide empirical evidence on the achievement and 
attainment of the desired behaviors, balancing the defined 
measurement categories, and prioritizing the measurements 
for implementation based on the balance. 

Other features and advantages of this invention will 
become apparent from the following detailed description of 
the presently preferred embodiment of the invention, taken 
in conjunction with the accompanying drawings. 

Brief Description of the Drawings 

Figure 1 is high level flow diagram illustrating the 
context of the method of a preferred embodiment of the 
invention. 

Figure 2 is a high-level overview of the measurement 
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and performance management (MAPM) method of a preferred 
embodiment of the invention. 

Figure 3 is a diagram detailing the high-level overview 
of the MAPM method of Figure 2 from a work product-based 
view. 

Figure 4 is a diagram illustrating the measurement gap 
analysis embodiment of the invention. 

Figure 5 is a diagram illustrating a measurement model 
for an IT organization including the notion of behaviors in 
accordance with a preferred embodiment of the invention. 



Best Mode for Carrying Out the Invention 

Referring to Figure 1, the process of the preferred 
embodiment of the invention for creating a measurement model 
work product 102 is part of the requirements phase of the 
measurement and performance management (MAPM) methodology. 
This process relies on input from a target future business 

END9 2000 0041 USl 6 



capabilities work product 101 and provides critical input 
into the measurement gap analysis work product 103. 
Measurement model 102 represents the core of requirements 
for the MAPM method and is used as the basis for developing 
process and organizational linkages, and many other 
technical work products to support its implementation with 
an information technology (IT) organization. A more 
detailed description of the process for creating measurement 
model work product 102 will be provided hereafter, 
particularly in connection with Figure 6. 

Referring to Figure 2, a high level view of the method 
of a preferred embodiment of the invention includes in step 
200 collecting into a competency-defined measurement 
categories and measurements file definitions of human 
resources 210, quality 211, customer 212, cost and schedule 
213, process 214, and productivity and output 215 
measurement parameters. In step 202, a measurement adoption 
method is executed, thereby selecting and defining contract 
measurements. In step 204, a measurement architecture and 
implementation method is executed to implement the contract 
measurements. In step 206, a measurement operations 
services method is executed to implement ongoing use and 
maintenance of measurements. As is represented by step 208, 
measurement program work products are produced and used 
END9 2000 0041 USl 7 



throughout steps 200-206. 



Referring to Figure 3, the system and method of the 
preferred embodiments of the invention are built upon the 
IBM Work Product-based Methodology Development Approach as 
defined and implemented by Solution Consulting Integration 
(SCI) practice through its SCI Enablement Process and new 
work products which extend the SCI method. These new work 
products include three work products representing the 
formulation of the measurement requirements that are to be 
implemented in an engagement, and are illustrated in Figure 
3 as measurement model 102, current customer measurements 
242, and measurement gap analysis 103, These new work 
products also include four work products representing 
additional implementation and/or technical requirements to 
support the implementation of measurement model 102, and 
include interface agreement 268, configuration script 272, 
contract measurement business policy 27 6, and scorecard 270, 

Practitioner accesses to the documents that are the 
work products of the preferred embodiment of the method of 
the invention may be, for example, by way of HTML code that 
provides users of the method access to work product (WP) 
dependency diagrams, engagement models, work product 
description papers and work product technique papers. 
END9 2000 0041 USl 8 



Similarly^ such documents may be maintained in collaboration 
space, and available based on access controls, to members of 
development, marketing and implementation teams throughout 
an enterprise. The IBM QuickPlace (TM) provides such a 
collaboration space. The Lotus Domino/Notes (TM) may also 
be used to control access using access control lists and 
provide consistency among several instances of data using 
replication. 

An MAPM engagement includes three contextual phases 
220, 222, and 224. Proposal phase 220, in the context of 
developing a measurement solution, represents the 
development of the initial solution specifications, 
including costs, that are to be delivered to a customer. 
Due diligence phase 222 represents the validation of 
assumptions and expectations that are made in developing a 
proposal to the customer before an outsourcing agreement is 
signed. Transition/transformation phase 224 represents the 
first major group of activities that occur once a legal 
contract for outsourcing has been established. This is 
where the transfer of resources and assets occurs and the 
implementation of specific new processes and procedures is 
introduced into this new organization. 



Proposal phase 220 includes the following work 
END9 2000 0041 USl 9 



products: current customer measurements 242, target future 
business capabilities 101, future process design points 248, 
to-be organization design 250, measurement model 102, 
recoitonendation and preliminary business case 240, and 
current IT infrastructure 230. Other work products 
providing input to proposal phase 220 include measurement 
catalog 234 and business policy 236. 

Due diligence phase 222 includes the following work 
products: measurement gap analysis 103, to-be process design 
252, measurement model 102, interface agreement 268, model 
of network workstation technical architecture 260, current 
application portfolio 2 64, and model of technical server 
architecture 262. 

Tr^najJjL^BT^^brai^^ phase 222-~±n clud cs~~th e^^ 

following work products: interf aceagj:::eemSnt 268, deployment 
plan 274, external interfac^^'^quirements 266, configuration 
script 272^p-'eti3^ser training materials 278, contract 

As is represented by line 401, current IT 
infrastructure work product 230 provides input to model of 
network workstation technical architecture work product 260. 
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As is represented by line 402, measurement model work 
product 102 provides input to architecture work product 260, 

As is represented by line 403, technical architecture 
work product 260 provides input to model of technical server 
architecture 262. 

As are represented by lines 404, 405, and 406, inputs 
to measurement gap analysis work product 103 come from to^be 
organization design work product 250, future process design 
points work product 248, and current customer requirements 
work product 242, respectively, with feedback to to-be 
organization design work product 250 from measurement gap 
analysis work product 103. 

As is represented by line 407, measurement model work 
product 102 provides input to recommendation and preliminary 
business case work product 240. 

As is represented by line 408, measurement gap analysis 
103 and measurement mode 102 feed each other. 

As is represented by line 409, target future business 
capabilities work product 101 provides input to measurement 
model 102, as further described above in connection with 
END9 2000 0041 USl 11 



Figure 2 . 



As is represented by lines 410, 411, 412 and 415, 
measurement model work product 102 provides inputs to model 
of technical server architecture work product 262, current 
application portfolio work product 264, configuration script 
work product 272 and interface agreement work product 268, 
respectively. 

As is represented by lines 413 and 414, measurement 
model work product 102 receives inputs from measurement 
catalog work product 234 and business policy work product 
236, respectively. 

As is represented by lines 417 and 423, current 
application portfolio work product 264 provides inputs to 
interface agreement work product 268 and contract 
measurement business policy work product 276. 

As is represented by lines 418 and 424, interface 
agreement work product 268 provides inputs to configuration 
script work product 272 and scorecard 270, respectively. 

As is represented by line 419, external interface 
requirements work product 266 provides input to interface 
END9 2000 0041 USl 12 



agreement 2 68. 

As is represented by lines 420 and 427, configuration 
script 272 provides inputs to scorecard 270 and contract 
measurement business policy work product 276, respectively. 

As is represented by line 421, model of technical 
server architecture work product 262 provides input to 
deployment plan work product 274. 

As is represented by line 422, end-user training 
materials work product 278 provides input to deployment plan 
work product 274. 

As is represented by line 425, contract measurement 
business policy work product 27 6 provides input to end-user 
training materials work product 278. 

As is represented by lines 426 and 428, model of 
network workstation technical architecture work product 260 
provides inputs to contract measurement business policy work 
product 276 and deployment plan work product 274, 
respectively. 



As is represented by line 429, scorecard work product 
END9 2000 0041 USl 13 



270 provides input to end-user training materials 278, 

As is represented by line 430, measurement gap analysis 
work product 103 and to-be process design work product 252 
exchange inputs and outputs, 

5 Current IT infrastructure 230 describes the physical 

systems and network configurations of the customer. 

Target future business capabilities 101 defines the 

^ customer's goals and objectives, necessary to achieve 

yj through measurements. 

U 10 Measurement catalog 234 is the intellectual capital 

O that contains the results of all previous engagements, and 

0 is used to support new engagements in the selection of 

O measurements . 

Business policy 236 represents optional intellectual 
15 capital describing business policies. 

Measurement model 102 defines the key requirements for 
the engagement, represents the translation of target future 
business capabilities 101 into account specific behaviors, 
and includes measures that empirically demonstrate those 
END9 2000 0041 USl 14 



behaviors. The process implemented by measurement model 102 
will be further described hereafter in connection with 
Figures 5 and 6. 

Recommendation and preliminary business case 240 
defines high level cost benefit estimates on the technical 
solutions that might be considered for measurement model 
implementation . 

Current customer measurements 242 provides a detailed 
list of the measurements that are being collected and 
reported currently by the customer • 

Future process design points 248 provides high level 
requirements for developing the to-be process design 252. 

To-be organization design 250 is the planned 
organization model. 

To-be process design is 252 is the planned processes 
model . 

Model of network/workstation technical architecture 260 
is a systems view of the hardware and software that supports 
the end users of the measurement system. 
END9 2000 0041 USl 15 
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Model of technical server architecture 262 is a systems 
view of the hardware and software server that supports the 
measurement system implementation and the system' s end 
users . 

Current application portfolio 264 is a detailed list of 
the applications that are being supported by the 
vendor/service organization. 

External interface requirements 266 is a detailed list 
of the outside systems that the measurement system must 
interface in order to perform data collection or data 
extraction. 

Interface agreement 268 is a formal agreement between 
systems to ensure that expectations for data collection are 
appropriately documented. 

Scorecard 270 is the visualization (reporting) 
requirements of the measurement system. 

Configuration script 272 is the code necessary to 
configure the tools used to implement the measurement model. 



Deployment plan 274 is the project plan for 
END9 2000 0041 USl 16 



implementing the system in the organization. 

Contract measurement business policy 27 6 provides the 
basic expectations of behavior on the account that must be 
conformed to in order to support data collection, reporting 
and analysis for the measurement system. 

End user training materials 278 are educational 
materials used to teach the organization about the 
measurement system and its supporting tools. 

Developing a system that measures the performance of an 
IT organization is critical to managing the IT organization 
and continually improving the processes that it employs over 
time. In accordance with a preferred embodiment of the 
invention, an IT measurement system is aligned with the IT 
organization's business goals by examining the IT 
organization's current state-of-af fairs with respect to 
measurement and analyzing how it maps to a proposed 
measurement model. To ensure the viability of the proposed 
measurement model, it is important the IT organization be 
structure to support the model and that the processes 
employed within the organization lend themselves to 
naturally support the collection, analysis, reporting, and 
use of the measurements in the proposed model. In 
END9 2000 0041 USl 17 



accordance with a preferred embodiment of the invention, a 
method is provided for systematically analyzing how the IT 
organization's current measurement system, if one exists, 
maps to the proposed measurement model and for identifying 
impacts to the to-be organization design and the to-be 
process design. 

Measurement gap analysis 103 defines the difference 
between the measurement model 102 and current customer 
measurements 242, additionally, with reference to to-be 
organization design 250 and to-be process design 252, 
considering possible deficiencies in the organization model 
and organization processes, 

Heretofore organizations in industry, while recognizing 
the need for establishing or improving measurement systems, 
have done so through a project-level perspective. In 
accordance with the system and method of the preferred 
embodiment of the invention, measurement gaps are addressed 
at the contract or program level. The gaps which are 
identified include gaps in the measures to be collected, 
analyzed, reported, and used, together with the critical 
relationship of measurements to the underlying organization 
structure and the processes that the organization employs. 
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Referring to Figure 4, a flow diagram of the 
measurement gap method of a preferred embodiment of the 
invention illustrates the inputs and outputs and the steps 
executed in developing measurement gap analysis work product 
103. 

In step 302, current customer measurements 242 are 
reviewed. 

In step 304, current customer measurements 242 are 
mapped to measurement model 102. 

In step 306, measurement gaps are identified. 

In step 308, measurements not covered by the 
measurement model are identified. 

In step 310, non-productive measurements are 
identified. 

In step 312, the impact to the existing organizational 

structure is identified. 

In step 314, the impact to processes is identified. 
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In step 316^ other requirements for measurement program 
success are identified. 

The canonical method of the preferred embodiment of the 
invention for developing a system that measures the 
performance of an IT organization involves identifying 
measures for an IT organization that are directly tied to 
goals and behaviors that the IT organization should be 
exuding, as defined by the IT management team, ''Canonical" 
means fully thought through and consistent. The measurement 
model work product 102 and associated techniques work from a 
top-down approach, starting with the goals of the 
organization, identifying appropriate behaviors and finally 
associating measures that will empirically demonstrate 
achievement of the behaviors. In addition, algorithms for 
prioritization are executed to ensure that the best fitting 
measures are selected in the final measurement model. 

Referring to Figure 5, this canonical process for 
defining measurements model work product 102 includes in 
step 232 articulating the envisioned to-be business goals 
through a series of goals and questions or behaviors. In 
step 236, in an optional, more specific embodiment, standard 
business policy driving behaviors that are assiimed or 
expected are defined. In step 234, a measurements catalog 
END9 2000 0041 USl 20 



of industry measurements are enumerated and fully defined in 
a database of existing contract measurements, including 
behaviors and goals that they satisfy. 



In step 326, an intellectual mapping is done between 
5 the envisioned to-be business goals and potential measures. 

A prioritization and balancing process is applied to ensure 
that the best measures are selected. This step 325 involves 
a two step prioritization and balancing process including, 
in step 328, prioritization of measures by number of 
10 objectives and behaviors satisfied, and in step 330 

identification of measures that have the highest number of 
occurring related measures in the same or another category. 

The method of the preferred embodiment of the invention 
draws upon several concepts from goals-question-metric (GQM) 

15 techniques described in Basili, Victor et al., ''The Goal 

Question Metric Approach, " Institute for Advanced Computer 
Studies, Department of Computer Science, University of 
Maryland, College Park, Maryland. To this are added notions 
of behaviors as a bridge between organizational goals and 

20 the identification and definition of measures that enable 
the required behaviors attempting to occur. The inclusion 
of behaviors in the overall technique implemented by the 
preferred embodiment of the invention creates a more tightly 
END9 2000 0041 USl 21 



integrated measurement model especially when dealing with 
measurement definition at the IT organization level, where 
business drivers and other organizational objectives form 
the basis of measurement requirements. Behaviors are the 
performance expectations of the individuals affected by the 
measurement model. These could be new ways of acting or new 
areas of focus in the respective roles on the account. 

rfttfi ^i-r-ing F-jfjnrn — a uiore de^ilcd dcGcr iption o^ 
this canonical process includes, in step 340, d^Jdrning the 
IT organization goals; in step 341, defipir^^g associated 
behaviors; in step 342, mapping beh^^iors against the 
measurement catalog; in steo^^3C3, building a first draft 
measurement model; in s;tep 344 prioritizing measures by 
number of behavioj^""^ satisfied; in step 345, prioritizing 
measures on mlfaber of related measures; in step 346 building 
a second/draft measurement model; and in step 347 performing 
gaiv^alysis. Each of these steps will be further 
dlesiiuzibgd^. 

In step 340, IT organization goals are identified 
during the creation of target future business capabilities 
work product 101. Typically, the goals of the organization 
are broken out into one or more categories of measures. In 
accordance with the preferred embodiment of the invention, 
END9 2000 0041 USl 22 



these six categories include those previously discussed in 
connection with Figure 2, and are human resources 210, 
quality 211, customer 212, cost and schedule 213, process 
214, and productivity and output 215, 

In step 341, behaviors that best demonstrate 
achievement of a particular organizational goal are defined. 
For each of the organization goals defined in target future 
business capabilities work product 101, one or more 
behaviors are defined. Similar in concept to the question 
aspects of the GQM approach, behaviors are defined to 
represent those characteristics the IT organization should 
foster as part of its normal operations. Behaviors identify 
an end state that should be directly tied to the 
organizations goals . 

As an example, if the organization's goal is in the 
area of ensuring that appropriate processes exist in the IT 
department, several behaviors might be identified, such as 

(1) the ability of the organization to forecast process 
needs, that is processes that should exist today but do 
not); (2) the knowledge of the IT staff of existing 
processes; (3) the ability to manage to process gaps; and 

(4) the consistent use and adherence to documented 
processes. 

END9 2000 0041 USl 23 



Behaviors are typically simple sentence structures and 
may even be incomplete sentences. They are short statements 
that represent ideas, actions or activities that demonstrate 
commitment and refinement of the organization goals. 
Typically, the organization goals are expressed as high 
level statements or highly summarized notions. Behaviors 
are intended to be a level or two below the organization 
goals and bring about a level of granularity low enough to 
support identification of measures that might empirically 
demonstrate achievement of the desired behaviors. 

A sample table for capturing the results of this step 
341 is included in Table 1: Goals, Behaviors, and 
Measurements : 





TABLE 1: 


GOALS, BEHAVIORS, 


AND MEASUREMENTS 


Goal 


1 


Behavior 1 


Measurement 1 


Goal 


2 


Behavior 2 


Measurement 2 



where measurements are those identified in step 342, infra. 

In step 342, those measurements from the measurement 
catalog 234 that best satisfy the identified behaviors are 
identified. This catalog is a database accessed using 
associated behaviors as a search parameter. 
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In step 343, the first draft 326 of measurement model 
102 is built. This involves documenting the results of step 
342 in the measurements column of Table 1 and in 
measurements model 326. In addition, other attributes of 
the measures are captured in measurement model 326, with 
their base values coming from measurement catalog 234. 
Table 2 illustrates the fields in measurements model 326 for 
capturing measurements and other attributes. 
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TABLE 


2: MEASUREMENTS MODEL FORMAT 






FIELD NAME 


DESCRIPTION 






MEASUREMENT NAME 


NAME OF THE MEASURE 




5 


MEASUREMENT 
DEFINITION 

ELEMENTAL METRICS 


ITS DEFINITION 

WHAT IS THE ELEMENTAL DATA REQUIRED TO 
CALCULATE THE MEASURE? 




10 


MEASUREMENT 
CALCULATION 


HOW IS THE MEASURE CALCULATED? 






RELATED 
MEASUREMENTS 


WHAT RELATED MEASURES EXIST FOR THIS 
MEASUREMENT? 


y - 
hi 




FREQUENCY OF DATA 
CAPTURE 


HOW OFTEN IS DATA FOR THIS MEASURE 
COLLECTED? 


?? S 

yj 


15 


FREQUENCY OF 

MEASUREMENT 

REPORTING 


HOW OFTEN ARE THE RESULTS OF THIS 
MEASURE REPORTED? 


p 




REPORTING PROCESS 


WHAT I.T. PROCESS TYPICALLY REPORTS THIS 
DATA? 


y 

Q 


20 


EVENT TRIGGER 


WHAT EVENT OR STEP IN THE PROCESS 
TRIGGERS THE AVAILABILITY OF THIS DATA 






ORGANIZATIONAL 
OWNER 


NAME OF ORGANIZATIONAL OWNER 



In step 344, a first prioritization exercise 328 is 
25 performed on the measures in the first draft 326 of 

measurement model 102. For each of the measures identified 
in step 342, a table is built that describes the 
relationship with each of the behaviors it may satisfy. In 
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many cases, a single measure may actually satisfy more than 
one behavior. This is important in the identification and 
selection of measures, since it is most desirable to keep 
the final number of measurers to a minimum. This 
prioritization process is done for each of the categories of 
the measurement model 102. Table 3, First Prioritization 
Example: People Measurements, illustrates how to capture the 
prioritization in this step 344. 
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TABLE 3: FIRST PRIORITIZATION EXAMPLE: PEOPLE MEASUREMENTS 
PART A 

BEHAVIORS /MEASUREMENTS 



PEOPLE BEHAVIORS 



JOB 

SATIS- 
FACTION 



TURNOVER 
RATE 



REASONS 
FOR EXIT 



CERTI- 
FICATION 
PROGRAM 
EFFECTIVE- 
NESS 



FORECAST SKILLS NEED ON 
A ROLLING X MONTH BASIS 



X 



KNOW/MONITOR THE 
CURRENT SKILLS 



X 



MANAGE TO THE SKILLS 
GAP 



ACTIVELY MANAGE 
ENVIRONMENTAL ISSUES 



X 



X 



PLAN AND MANAGE TO 
SHORT-TERM AND LONG-TERM 
USAGE OF CONTRACTORS 

MANAGE TO THE REDUCTION 
OF LONG-TERM CONTRACTOR 
DEPENDENCIES 

RECRUIT ENTRY LEVEL 
STAFF TO MEET A PORTION 
OF FUTURE SKILLS NEED 

HIRE EXPERIENCED PEOPLE 
TO FILL IMMEDIATE SKILL 
NEEDS, WHICH ARE LONG- 
TERM IN NATURE 

PROACTIVELY MANAGE 
RECRUITMENT UNTIL NEW 
HIRES ARE ON-BOARD 

PROACTIVELY FOSTER LONG- 
TERM RELATIONSHIPS WITH 
LOCAL RECRUITMENT 
SOURCES 

CREATE AN ENVIRONMENT 
ATTRACTIVE TO TARGETED 

RECRUITS X X 

MEASUREMENT SUBTOTAL 2 2 2 0 
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TABLE 3: FIRST PRIORITIZATION EXAMPLE: PEOPLE MEASUREMENTS 
PART B 

BEHAVI ORS /MEASUREMENT S 



5 PEOPLE BEHAVIORS CONTRACTOR CONTRACTOR UNFILLED JOB OFFER 

ENGAGEMENT USAGE POSITIONS ACCEPTANCE 

DURATION RATE 



FORECAST SKILLS NEED ON 
10 A ROLLING X MONTH BASIS 

KNOW/MONITOR THE 

CURRENT SKILLS XXX 
MANAGE TO THE SKILLS 

GAP X X 

15 ACTIVELY MANAGE 

ENVIRONMENTAL ISSUES 

^ PLAN AND MANAGE TO 

y= SHORT-TERM AND LONG-TERM 

LlJ USAGE OF CONTRACTORS X X 

n 3 

y 20 MANAGE TO THE REDUCTION 

^ OF LONG-TERM CONTRACTOR 

2 DEPENDENCIES X X 

- RECRUIT ENTRY LEVEL 

O STAFF TO MEET A PORTION 

m 25 OF FUTURE SKILLS NEED X 

rl HIRE EXPERIENCED PEOPLE 

Jj' TO FILL IMMEDIATE SKILL 

y NEEDS, WHICH ARE LONG- 

O TERM IN NATURE X X 

30 PROACTIVELY MANAGE 

RECRUITMENT UNTIL NEW 
HIRES ARE ON-BOARD 

PROACTIVELY FOSTER LONG- 
TERM RELATIONSHIPS WITH 
35 LOCAL RECRUITMENT 

SOURCES 

CREATE AN ENVIRONMENT 
ATTRACTIVE TO TARGETED 
RECRUITS 

40 HE^lTRE^flEHT SUBTOT^ 4 4 4 
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Th6 measurement subtotal scores in Table 3 are derived 
from simple counts of the numbers of X's in each of the 
columns. Those measures with the highest scores satisfy the 
most number of behaviors and are more desirable as measurers 
5 in the model since they provide more knowledge and insight 
for the cost of implementation. Of course, the table could 
be modified to reflect weighting factors on both cost and 
ef f ectiveness/ but such is not deemed essential for this 
embodiment of the invention. 



^ 10 In step 345, the second prioritization process 330 is 

yl 

yj performed on the measurements in the first draft 326 of 

nJ 

y measurement model 102. For each of the measures identified 

ass 

U in step 342, a table is built that describes the related 

B 

O measure. In many cases a single measure may actually be 

O 15 related to more than one measure. This is important in the 

yj. 

□ identification and selection of measures, since it is most 

desirable to keep the final number of measures to a minimum. 
This prioritization process 330 is done for each of the 
categories of the measurement model. Table 4, Second 
20 Prioritization Example: People Measurements, illustrates how 
to capture the prioritization in this step 345. 



END9 2000 0041 USl 



30 



10 



15 



OJ 



20 



TABLE 4: SECOND PRIORITIZATION EXAMPLE: PEOPLE MEASUREMENTS 

PART A 



PEOPLE MEASUREMENTS 



JOB 
SATIS- 
FACTION 



JOB SATISFACTION 0 
TURNOVER RATE 

REASONS FOR EXIT X 

CERTIFICATION 
PROGRAM 

EFFECTIVENESS X 
CONTRACTOR 

ENGAGEMENT DURATION X 

CONTRACTOR USAGE X 

UNFILLED POSITIONS X 

JOB OFFER 
ACCEPTANCE RATE 

MEASUREMENT SUBTOTAL 5 



PEOPLE MEASUREMENTS 



TURNOVER 
RATE 



REASONS 
FOR EXIT 



X 
0 



X 



X 
X 
0 



CERTI- 
FICATION 
PROGRAM 
EFFECTIVE- 
NESS 



X 
X 



X 
3 
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TABLE 4: SECOND PRIORITIZATION EXAMPLE: PEOPLE MEASUREMENTS 
PART B 

PEOPLE MEASUREMENTS 



PEOPLE MEASUREMENTS CONTRACTOR CONTRACTOR UNFILLED JOB OFFER 

ENGAGEMENT USAGE POSITIONS ACCEPTANCE 

DURATION RATE 



JOB SATISFACTION X 

TURNOVER RATE X X 

REASONS FOR EXIT 

CERTIFICATION 

PROGRAM 

EFFECTIVENESS 

CONTRACTOR 

ENGAGEMENT DURATION 0 X 



CONTRACTOR USAGE 0 X X 

UNFILLED POSITIONS 0 X 
JOB OFFER 

ACCEPTANCE RATE XX 0 

MEASUREMENT SUBTOTAL 2 3 2 2 



The measurement subtotal is derived from the sum of X' s 
in each of the columns. Those measures that have the 
highest scores also have the highest number of related 
measures, making them desirable as measurement for the base 
model. When prioritized against the results of the 
behaviors to measurement matrix (Table 3) , an initial set of 
measurements can be identified. The cells with a ^'0" entry 
indicate the same measure to same measure condition, which 
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should not be considered in the matrix, because it is not 
meaningful to relate a measure to itself. 

In step 346, the results of the prioritization 
processes 328 and 330 are used to build the second draft 332 
5 of measurement model 102. The tables built in steps 344 and 
345 are used to select those measures that should be 
included in the final measurement model. A simple approach 
is to calculate the mean values of measurement subtotals 
from each of Tables 3 and 4 and select those measures that 
10 have scores greater than or equal to that mean. 

As an example, in step 344, the mean score is 

[ (2 + 2+2+0+4 + 4+4+1) /8] = 2.375 

As a result, the following measures would be selected: 
contractor engagement duration, contractor usage, and 
15 unfilled positions. Some measures may need to be included 

in measurement model 102 in exception of this prioritization 
process either because of specific customer need or because 
it may be the only measure that satisfies a particular 
behavior and/or goal. 



20 



The selected measures are then captured in the format 
END9 2000 0041 USl 33 



described in step 343, Table 2. 

In step 347, once the final set of measures has been 
dociimented, a gap analysis 103, as previously described in 
connection with Figure 4, needs to be done to ensure that 
linkages are in place between the measurements, processes 
and organization. In addition, any time a change that 
results in the addition or deletion of a measurement in the 
model occurs, the gap analysis step is performed. 

Advantages over the Prior Art 

It is an advantage of the invention that there is 
provided a system and method for systematically defining and 
implementing a measurement program. 

It is a further advantage of the invention that there 
is provided a system and method for implementing a 
measurement program that is supported by an appropriate 
organizational structure and that is fully integrated with 
the organization's processes. 

It is a further advantage of the invention that there 
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is provided a system and method for identifying and defining 
measurements necessary to support the management of an IT 
organization. 

It is a further advantage of the invention that there 
is provided a system and method for linking defined 
measurements to organization goals and objectives. 

It is a further advantage of the invention that there 
is provided a system and method of integrating measurement 
ownership into an organizational model. 

It is a further advantage of the invention that there 
is provided a system and method for integrating measurement 
data collection into an IT process model. 

It is a further advantage of the invention that there 
is provided a system and method for developing a technical 
environment for supporting measurement data collection, 
analysis and reporting. 

It is a further advantage of the invention that there 
is provided a system and method for developing the technical 
interfaces necessary to support measurement data collection. 
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# # 

It is a further advantage of the invention that there 
is provided a system and method for developing measurement 
reporting methods. 

It is a further advantage of the invention that there 
is provided a system and method for developing training and 
end-user support materials for a measurement system. 



Alternative Embodiments 



It will be appreciated that, although specific 
embodiments of the invention have been described herein for 
purposes of illustration, various modifications may be made 
without departing from the spirit and scope of the 
invention. In particular, it is within the scope of the 
invention to provide a computer program product or program 
element, or a program storage or memory device such as a 
solid or fluid transmission medium, magnetic or optical 
wire, tape or disc, or the like, for storing signals 
readable by a machine, for controlling the operation of a 
computer according to the method of the invention and/or to 
structure its components in accordance with the system of 
the invention. 
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Further, each step of the method may be executed on any 
general computer, such as an IBM System 390, AS/400, PC or 
the like and pursuant to one or more, or a part of one or 
more, program elements, modules or objects generated from 
any programming language, such as C++, Java, Pl/1, Fortran 
or the like. And still further, each said step, or a file 
or object or the like implementing each said step, may be 
executed by special purpose hardware or a circuit module 
designed for that purpose. 

Accordingly, the scope of protection of this invention 
is limited only by the following claims and their 
equivalents . 
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